Tell ptxas the largest block size a kernel can be launched with - #9284
Open
abadams wants to merge 3 commits into
Open
Tell ptxas the largest block size a kernel can be launched with#9284abadams wants to merge 3 commits into
abadams wants to merge 3 commits into
Conversation
Without it ptxas has to assume a block could hold the maximum number of threads, which caps it at 65536/1024 = 64 registers per thread. The kernel in apps/cuda_mat_mul is launched with 16x2 threads and wants 70 registers, so it spilled: 20 bytes of spill stores and 16 of spill loads. Giving ptxas the bound removes the spills, and the app goes from 0.314 ms to 0.233 ms. This has to be a function attribute. The equivalent nvvm.annotations entry, which is how the kernel annotation just above is written, is upgraded to one of these when a module is read from a file, but that never happens to a module we built ourselves, and the backend only looks at the attribute. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
alexreinking
approved these changes
Aug 5, 2026
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #9284 +/- ##
==========================================
- Coverage 70.32% 70.28% -0.05%
==========================================
Files 258 258
Lines 79160 79178 +18
Branches 18997 19002 +5
==========================================
- Hits 55672 55652 -20
+ Misses 17802 17801 -1
- Partials 5686 5725 +39 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Without it ptxas has to assume a block could hold the maximum number of threads, which caps it at 65536/1024 = 64 registers per thread. The kernel in apps/cuda_mat_mul is launched with 16x2 threads and wants 70 registers, so it spilled: 20 bytes of spill stores and 16 of spill loads. Giving ptxas the bound removes the spills, and the app goes from 0.314 ms to 0.233 ms.
I believe this is the underlying reason we found a benefit in manually messing with the number of registers with an env var - code that was backed out in #9260.